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ABSTRACT  - J 

Absolute  continuity,  process  representations,  and  the  Shannon  information 
are  considered  for  problems  involving  a  Gaussian  mixture  process  (N-  ),  t  in 
[0,!].  N(u,t)  =  A(u)G(cj,t)  a.e.  dP(u)dt,  where  (Gt)  is  a  Gaussian  process  ana 
A  is  a  positive  random  variable  independent  of  (Gt).  Let  (Y  ) .  t  in  [0,1],  be 

t  " 

a  second  process  with  Uy  and  the  measures  induced  on  IR^’^  and  Py  and  p^ 
the  measures  induced  on  LgCO.l]  (when  (Y ^ )  has  paths  a.s.  in  ^[O.l]).  The 
Cramer-Hida  spectral  representation  and  an  extension  of  Girsanov’s  theorem  are 
used  to  obtain  results  on  absolute  continuity  (uy  <<  and  Py  <<  p^)  and 
likelihood  ratio  in  terms  of  similar  results  involving  a  Gaussian  mixture 
local  martingale,  for  which  representations  are  given.  These  results  are  then 
applied  to  obtain  the  Shannon  mutual  information  for  a  communication  channel 
with  feedback  having  (N^)  as  additive  noise.  Capacity  is  obtained  for  the 
no-feedback  channel,  subject  to  an  average-energy  type  of  constraint. 
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0.  Introduction 


Let  I  denote  the  interval  [0,1].  (N t )  will  denote  a  real  valued 
stochastic  process  having  index  set  I  and  representation  (AGt),  where  (G^)  is 
a  mean-square  continuous  Gaussian  process  with  index  set  I  and  mean  zero,  and 
A  is  a  positive  real  random  variable,  independent  of  (G^).  Let  (Y^)  be  a 
second  stochastic  process  with  index  set  I.  The  measures  induced  by  (Y^)  and 
(N^)  on  !R*  are  denoted  i>y  and  v ^  respectively.  The  problem  first  considered 
here  is  that  of  determining  conditions  for  Uy  to  be  absolutely  continuous  with 
respect  to  (denoted  i>y  <<  u^). 

The  class  of  processes  (N t )  having  the  above  representation  is  quite 

large,  containing  net  Gaussian  processes,  but  also  those  that  are 

spherically  invariant  [13]  and  a  large  class  of  a-subGaussian  processes  [7]. 

2 

In  the  a-subGaussian  case,  A  is  required  to  be  a/2 -stable,  and  in  the 

2 

spherically  invariant  case  EA  is  required  to  be  finite.  We  define  (N^)  to  be 
a  Gaussian  mixture  process  and  a  Gaussian  mixture  measure. 

Since  G  is  mean-square  continuous,  almost  all  paths  of  (Nt)  belong  to 

and  we  also  consider  the  absolute  continuity  problem  for  measures  My 
and  p^  induced  on  L2[0,l].  When  absolute  continuity  holds,  we  obtain  the  Radon- 
Nikodym  derivative  dpy/dp^. 

Conditions  for  absolute  continuity  and  expressions  for  the  Radon-Nikodym 
derivative  when  N  is  Gaussian  are  given  in  [4]. 

The  paper  has  been  organized  to  match  the  ideas  structuring  the  methods 
used  in  obtaining  its  results.  The  Cramer-Hida  representation  [8,11]  allows 
one  to  consider  the  noise  as  the  summed  output  from  a  set  of  causal,  non-anti¬ 
cipating  and  non-random  linear  filters,  each  of  which  is  driven  by  a  "white 
noise"  process.  Assuming  that  the  received  signal  has  paths  that  are  in  the 

•*  • 
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reproducing  kernel  Hilbert  space  (RKHS)  of  the  noise,  which  means  that  they 
are  smoother  than  those  of  the  noise,  one  can  represent  the  received  signal  as 
summed  outputs  of  the  same  set  of  linear  filters,  provided  the  vector  of  input 
signal-plus-noise  processes  has  a  law  that  is  absolutely  continuous  with 
respect  to  "white  noise"  [4].  Thus  the  problem  of  the  absolute  continuity  of 
the  inputs  (vector  processes)  is  first  solved  in  Section  1  and  Section  2,  and 
that  for  the  outputs  in  Section  3. 

The  work  on  absolute  continuity  w.r.t.  a  Gaussian  mixture  has  at  least 
two  major  applications.  One  is  to  detection  of  signals  imbedded  in  noise  of 
this  type;  see  [5],  [6],  [21]  for  a  discussion  of  such  problems.  The  second 
application  is  to  obtain  a  complete  and  detailed  derivation  of  the  (Shannon) 
mutual  information  in  an  additive  feedback  channel  when  the  noise  is  a 
Gaussian  mixture.  Even  for  the  Gaussian  channel,  such  a  derivation  does  not 
exist,  although  the  groundwork  has  been  laid  in  papers  by  Kadota,  Zakai .  and 
Ziv  [15]  and  by  Hitsuda  and  Ihara  [12].  We  have  previously  stated  this  result 
without  proof  [3].  Section  4  contains  a  detailed  derivation.  Capacity  of  the 
no-feedback  channel  follows  immediately  from  this  expression  and  a  lower  bound 
given  in  [3],  An  information  channel  is  said  to  be  mismatched  [2]  if  the 
constraint  on  the  transmitted  signal  is  not  given  directly  in  terms  of  the 
covariance  of  the  ciianncl  noise.  Such  channels  constitute  the  usual  case  in 
practical  problems.  Although  the  capacity  problem  has  been  solved  for 
mismatched  no-feedback  Gaussian  channels  [2],  the  solution  is  obtained  by 
Hilbert  space  methods  that  do  net  easily  carry  over  to  time-continuous 
feedback  channels.  The  stochastic  process  formulation  given  here  seems  better 
united  to  the  feedback  channel.  Capacity  of  mismatched  channels  with  feedback 
comprises  one  of  the  major  areas  of  open  problems  in  fundamental  information 
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theory;  even  for  the  Gaussian  time-discrete  channel,  a  general  solution  has 
not  been  obtained.  Thus,  it  seems  particularly  important  that  a  complete  and 
general  development  of  the  expression  for  mutual  information  be  available. 

All  stochastic  processes  will  be  defined  on  a  complete  probability  space 
(Q.S.P)  and  will  have  index  set  I,  unless  otherwise  stated.  A  stochastic 
process  whose  law  is  determined  by  P  will  be  written  in  simple  form,  e.g. , 
(V^).  If  the  law  is  determined  by  a  probability  Q  on  (9,0),  then  the  process 
will  be  denoted  by  (V*j|).  For  a  stochastic  process  (V^),  cP(V)  will  denote  the 
o-field  generated  by  {Vs>  s£t},  and  ot(V)  its  P-completion.  o^(V)  and  o(V)  are 
the  corresponding  filtrations,  for  example:  o^(V)  =  (o^(V),  t€I}.  Lt(V)  is  the 
closed  linear  span  in  LgfP]  of  the  set  {Vs,  s£t}. 

For  a  positive  integer  M  <  <•>,  will  be  the  Borel  o-field  of  IR^  under 
the  product  topology.  is  the  set  of  all  the  real  valued  functions  defined 
on  I  that  are  continuous  and  vanish  at  t  =  0.  Cq  is  endowed  with  the  sup-norm 
topology,  and  <€g  is  the  resulting  o-field,  also  generated  by  the  evaluation 
maps  II Ht(f)  =  f(t),  t  €  I,  f  CCq.  The  Borel  o-field  of  is  <€^,  the 
product  o-field  of  M  copies  of  'Cq.  R*  is  the  space  of  real-valued  functions 
defined  on  I  and  the  o-field  in  R*  generated  by  the  cylinder  sets,  that  is, 
sets  of  the  form 

{x:  (x(t1) . x(tn))  €  n  ^  1  •  tj . tn  in  I.  An  €  &n. 

Let  (Vt)  have  paths  almost  surely  in  C^:  it  induces  a  measure  Py  on  <€^. 
For  a  scalar  process  (Vt),  Uy  Is  the  measure  induced  by  (V^.)  on  91* ,  and,  if  it 
is  measurable,  with  paths  almost  surely  in  the  measure  induced  on 

LgfO.l]  is  denoted  \jy.  V  will  denote  the  path  map:  V[u]  =  (V(w,t),  t  €  I},  or 

YM  =  (Y(w.t).  t  e  i}. 
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Proposition  1-  B  has  the  representation  B(u,t)  =  A(w)G(<j,t),  t  €  I,  P-almost 

surely,  with  G  a  continuous  Gaussian  local  martingale  independent  of  A  and 

such  that  <G>  =  T.  A  is  adapted  to  s^.  (§t)  Is  an  (W.P)  martingale  if  and  only 

2 

if  EA  <  «  and  an  Lg-mart ingale  if  and  only  if  EA  <  00 . 
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Remark.  In  the  Introduction,  we  have  begun  with  a  Gaussian  process  (Gt).  In 
this  section,  we  begin  with  a  vector  process  (Bt)  as  in  A4,  and  derive  a 
representation  in  terms  of  a  vector  process  (G^) . 

Proof :  Since  <B>  =  A^T  and  T(t)  is  positive  definite  for  t  >  0,  A^(u)  =  trace 

<B>(w, t)/trace  r(t),  so  that  A  is  s^-measurable. 

(Bt)  is  a  continuous  local  martingale,  so  that  <B>  is  continuous  [16]. 

Since  A  is  almost  surely  positive,  T  must  also  be  continuous.  Define  a  vector 

Drocess  (Gt)  such  that  G(<j,t)  is  in  [R^  by  G(w,0)  =  0  and  G(w,t)  =  B(u,  t)/A(w) , 

t  >  0.  t  €  I.  (Qt)  is  then  continuous  and  adapted  to  d.  To  prove  that  (G^)  is 

a  Gaussian  martingale  with  variation  process  T,  it  is  sufficient  to  show  that 

(Gt)  is  a  local  martingale  with  variation  process  T  [16],  which  is  equivalent 

to  establishing  that,  given  any  fixed  0  in  G_  :=  <0,G>  u  is  a  local 

9  R* 

martingale  with  variation  process  <0,T0>  ... 

~  tr 

0 

Let  now  {R  }  be  a  sequence  of  stopping  times  which  reduces  BQ.  and  define 
n  u 

the  following  stopping  times: 


S9(w)  :=  inf {t  >  0:  |B0(u,t)|  >  n}, 

T9(W)  :=  inf {t  ^  0:  |Gfl(«.t)|  >  n} . 

U9  :=  R9  a  S9  a  T9. 

n  n  n  n 


As  n  increases,  each  of  these  stopping  times  converges  almost  surely  to  1, 

since  B0  and  G0  are  each  almost  surely  path  continuous  on  the  compact  I.  Now 
U 

G0  is  bounded  by  n  and  thus  integrable.  To  see  that  it  is  a  martingale,  there 
are  two  cases  to  consider: 
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1.  0  <  s  <  t 


u 


if 


It  then  suffices  to  write  G0n  as  B0n/A  and  to  take  into  account  that 

0 


+  Un 

A  is  adapted  to  si 0,  and  B0  a  martingale. 

0  =  s  <  t 

One  must  then  use  a  limiting  argument.  Fix  u  such  that  0  <  u  <  t.  It 


follows  as  in  1.  that 

,0 


,0 


U  U 

E{G0n(-.t)Mo}  =  E{C0n(*.u)|rfo} 


IT 

But  G0  1  is  almost  surely  path  continuous  and  thus,  almost  surely, 

Un  Un 

lim  G_  (».u)  =  0.  Since  furthermore  G-.  is  uniformly  bounded, 

uiO  0  0 


U 


lim  E{G_n(*  ,u)  =  0,  almost  surely. 

ulo  9  0 


To  verify  that  <G_>  =  <0,r0>  u,  one  notes  that 
0  Ik 


u0  u0  u0 

{G0n( * . t)}2  -  <0.r(tAU0)0>^  =  {[B0n( * . t)]2  -  <B0>  n( * , t) }/A2. 


U0 

9  0  n 

Since  i  R^,  B0  is  a  uniformly  integrable  martingale  for  fixed  n.  Since  A 

is  adapted  to  ai^,  the  right  side  of  the  last  equality  is  a  martingale. 

To  check  that  A  is  independent  of  G,  fix  0  <  s  <  tj  <  .  .  .  <  t  ,  and  let 

n 

I(a,a. . a  )  :=  E  exp[i{aA  +  2  <a,,G(*,t  )>  u}]. 

1  ~71  i=l  J  J  \ST 

Conditioning  on  si*  and  using  the  fact  that  (Gt)  Is  a  Gaussian  martingale,  one 
has : 

N 

I(a,a^ . a^)  :=  E  exp{ laA}  E  exp(i  2  <a1,G(*,t<)>  M}  n 


i=l 


j 


J'V' 
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Proposition  1  is  the  equivalent,  for  Gaussian  mixture  local  martingales, 
of  the  martingale  characterization  of  Brownian  motion.  So,  essentially  the 
same  proof  which  is  valid  in  the  Brownian  case  [9]  applies  to  Gaussian  mixture 
local  martingales  and  one  thus  has: 

Proposition  2-  Let  d^  ■=  Pq(B)  and  d^  :  =  (7^(2),  t  >  0  and  assume  that  =  5, 

T(t)  =  diag[I"j(  t) ,  .  .  .  t)] ,  t  €  I.  Then  every  local  martingale  (Vt)  for 

(sl.P),  with  paths  almost  surely  in  D[0,1],  has  the  representation 

M  t 

V(u.t)  =  V0(u)  +  2  /^(u.yJB^cj.dy), 

with  VQ  adapted  to  s<q,  each  (V i  t)  (^,P)-predictable  and  jQV^(u,y)ri (dy)  <  «° 
almost  surely  for  fixed  t  in  I. 


2.  Absolute  Continuity  With  Respect  to  a  Gaussian  Mixture  Local  Martingale 
(Bt)  is  the  Gaussian  mixture  local  martingale  defined  in  Section  1.  In 
this  section,  we  give  sufficient  conditions  for  P^  <<  Pg  and  ~  Pg  where  X 
is  a  second  M-vector  process,  and  also  give  am  expression  for  the  Radon- 
Nikodym  derivative  dPy/dP.,.  We  make  the  following  additional  assumptions. 


A5:  (X^Jis  an  M-component  vector  process,  separable  with  respect  to  closed 


sets . 


A6:  (st)  *s  311  M-component  vector  process,  ^-optional  and  such  that 

J"*<s(u,y) , r(dy)s(w,y)>  u  <  ®,  almost  surely. 

u  ur 

A6*:  Let  s«:  x  I  -»  IR^  be  ^-optional  (^t  :=  a-algebra  generated  by  the 

evaluation  maps  {II(c, s)  =  c(s),  s  £  t}),  and  such  that 
,Tq<s*(X[w]  ,y) ,  r(dy)s*(X[u]  .y))^  <  »,  almost  surely. 

A7-'  For  each  t  in  I,  X(<j,  t)  =  jQ<B>(w,dy)s(w,y)  +  B(<j,t),  almost  surely. 
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-  2  J'o<s(u.y).<B>(cj,dy)s(c..y)>^M}.  D(w)  : 
A8*:  Let  A  be  obtained  by  choosing  s*(X[u],t) 

it 

accordingly,  and  assume  that  ED  si. 


A(u,l).  and  assume  ED  =  1 . 
for  s(u,t)  in  A8,  D*  be  defined 


Proposition  3:  Let  dQ  =  DdP,  =  QoA  * ,  P^  =  P°A  * ,  =  QoX  P  =  P°B  1 

Then,  with  assumptions  A1-A8, 

a)  for  any  s^-measurable  random  variable  U  which  is  almost  surely  bounded 
with  respect  to  P.  E^U  =  EpU. 

b)  Qa  =  PA  and  0*  =  Pg. 


Proof"  A5  and  A7  imply  almost  sure  path  equality  in  A7,  so  that  the  paths  of 

(Xt)  are  almost  surely  continuous.  By  Girsanov*s  theorem  [16],  (Xt)  is  a 

Q  2 

continuous  (d,Q)~loca.l  martingale  such  that  <X>  s  A  7\  Consequently,  by 

Proposition  1,  (X^)  =  (AG^).  where  (G^)  is  a  continuous  (sf.Q) -Gaussian 
Q  0 

martingale  with  <G  >  =  T,  and  A  is  independent  of  (Qt)  with  respect  to  Q. 

Let  U  be  any  ^-measurable  random  variable  that  is  almost  surely  bounded 
with  respect  to  P.  Fix  t  >  0.  Then,  because  of  A8,  A  is  a  martingale  and 

EQU  =  Ep{UA( • . t)} . 


But  A  has  also  continuous  paths  and  lim  A(u.t)  =  1,  almost  surely  with  respect 

tio 

to  P.  Furthermore,  still  with  respect  to  P,  A(u>,t)  £  0  almost  surely,  and 
EA(*,t)  =  1,  for  all  t  in  I.  Consequently,  A { * , t )  -*  1  in  a(L.(P),L  (P))  as 
t  1  0  [9],  Thus: 

EqU  =  lim  Ep{UA( • . t)}  =  EpU. 
w  t  lo 
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The  equalities  and  =  Pg  follow  directly  from  this  result,  using 

i  0A 

characteristic  functions:  choose  first  e  for  U,  then  clearly,  P^  =  Q^.  Now 

n 

consider  E^V  (=  E„V) ,  where  V  =  exp{i  2  <g..X(*,t  .)>  „} .  Then  P.  =  Q.  yields: 
W  *  j_l  J  J  A  A 

n  n  0 

Eq  exp{i  2  <a  ,X( • , t  )>  }  =  Q  (da)EQ  exp{ia  2  <g  .GW(*,t  )>  } 

^  j_l  J  J  BT  ^0  A  4  j=l  J  RM 

2  n 

=  QA(da)  exp{-  |-  2  <g  ,T(t  )a  >  } 

J0  A  j=i  j  J  J  r” 

rf°  n 

=  PA(<ia)Ep  exp{ ia  2  <a  ,G(*,t  )>  } 

J0  A  P  j=l  j  J  RM 


n 


=  E_  exp{i  2  <a..B(*.t  )>  „} . 

j=l  J  J  RM 


Proposition  4: 

Let  -r(c.y)  :=  trace{2(c.y)ff(c,y) 1  -  2/^  ff(£.z)II(c,dz)t}/trace  T(y),  c  €  eg  and 
y  €  I  (ff(c.y)  :=  c(y) ) .  Then: 

a)  Al-A?  imply  P^  <<  Pg  and  A1-A8  imply  P^  ~  Pgi 

*  M 

b)  A1-A5  and  A6  -A8  imply  P^  ~  Pg  and,  with  respect  to  Pg,  for  almost  every 
c  in  eg. 

dPX  f1  «  f1  *  * 

jp“  (c)  =  exp{  <s  (c.y) ,2(c,dy)>  „  -  J  <s  (c.y) , <rp(c.dy)s  (c.y)>  }, 

d  B  J0  R*  J0  RM 

where  <II>(c,y)  =  "*(<:, y)f(y)  [20]. 


Proof :  The  result  follows  from  the  method  used  [19]  when  (B^)  is8  the  Wiener 
process,  together  with  Proposition  3. 
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Remark.  Sufficient  conditions  for  A8  and  A8  are  respectively  [19]: 


E  exp{£ 


<s(u. 

J0 


y)  ,  <B>((j,dy)s(w.y)>  „}  <  ®, 

nr 


PB< 


<s  (c.y) . <2>(c.dy)s  (c,y)>  M  <  °°}  =  1 . 
0  BT 


.1 


Assuming  that  E  ft.  <s(  •  ,y)  ,r(dy)s(  •  ,y)>  „  <  ®,  one  can  obtain  a  representation 
0  R* 

of  the  form  A7  starting  from  a  representation  of  the  form  A7.  These  results 
are  proved  as  in  the  Wiener  process  case  [19]. 


3.  Absolute  Continuity  With  Respect  to  a  Gaussian  Mixture  Process 

In  this  section  we  return  to  the  problem  discussed  in  the  Introduction, 
treating  a  Gaussian  mixture  process  (AG^).  where  (G ^ )  is  a  zero-mean  mean- 
square-continuous  Gaussian  process.  G  has  a  proper  canonical  representation 
[11],  which  we  assume  to  have  the  form 

G(u.t)  =  Xj<E(t.x).  G(cj,dx)>^M  . 

where  F  and  (G„)  have  respective  components  F.  and  (G  „ ) ,  1  <  i  <  M. 

Without  the  assumption  that  (Gt)  is  Gaussian,  the  processes  (G^  )  are 
zero-mean  orthogonal -increment  processe.  ,  mutually  orthogonal  and  mean-square 
continuous.  Their  non-decreasing  variances  EG^(*,t)  define  Borel  measures  T. 

I  in  the  usual  way:  f^(a,b]  =  EG^(’.b)  -  EG^(»,a);  moreover  r\  +  j  <<  f. . 


(1) 


on 


1  i  i  i  M-l.  Each  F^:  Ixl  -*  !R  is  Borel-measurable,  F^(t.x)  =  0  for  x  >  t,  and 
2  XqXqF^( t.s)ri(ds)dt  <  «  M  $  °®  is  the  multiplicity  of  G. 


i=l 


The  proper  canonical  representation  (1)  has  the  property  that 

-  *  j 

I_t(G)  =  Lt(G),  for  t  in  I.  and  that  I  XqF^ ( t . s)gj (s)!^ (dsl  =  0,  all  t  in  I, 
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if  and  only  if  =  0  in  1  £  i  £  M.  Thus  (F.(t,*),  t  €  1}  spans 

The  representation  (1)  is  an  equality  in  the  mean-square  sense,  thus 
holds  P-almost  everywhere  for  each  fixed  t  in  I .  However,  taking  both  sides  of 
(1)  separable  with  respect  to  closed  sets  means  that  we  can  assume  path 
equality  P-almost  everywhere. 

The  assumption  that  (Gt)  is  Gaussian  further  implies  that  {(G^  ),  i  <  M} 

are  a  Gaussian  family;  thus  the  the  component  processes  are  mutually 
independent .  have  independent  increments,  and  hence  can  be  assumed  to  be  path 

continuous.  Moreover  equality  of  Lt(G)  and  Lt(G)  implies  that  of  at(G)  and 

a  (G)  for  all  t  in  I.  Each  (G  ).  1  £  i  £  M,  is  a  martingale  with  respect  to 

t  i ,  t 

ct(G)  and  with  respect  to  g( G)  v  g(V),  where  V  is  any  stochastic  process 
independent  of  G. 

M  is  the  multiplicity  of  ( G t),  and  it  is  assumed  throughout  that  M  <  <». 
This  restriction  is  due  to  a  similar  restriction  in  [4],  on  which  part  of 
these  results  depend. 

We  assume  that  supportfP^]  =  and  that  support[p^]  =  LgCO.l].  Since 
these  measures  are  Gaussian,  their  supports  are  closed  linear  manifolds  equal 
to  the  closure  of  the  ranges  of  their  covariance  operators  [14].  One  can  thus 

always  work  with  this  subspace;  it  preserves  the  original  linear  space 

structure  under  the  original  norm  (and  inner  product,  for  L^CO.l]).  As  a 
consequence  of  these  assumptions,  one  can  take  (in  the  proofs)  each  f\  to  give 
positive  measure  to  each  interval  (a,b),  b  >  a. 

Let  V  be  a  mean  square  continuous  process:  its  reproducing  kernel  Hilbert 
space  (RKHS)  is  denoted  Hy.  and  its  covariance  by  Cy.  All  elements  in  Hy  are 
continuous  functions  defined  on  1.  (L  is  the  kernel  of  an  integral  operator  1L. 
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on  LgCO.l]  which  is  trace-class-'  the  covariance  operator  of  V,  Ry,  defines  a 

JL 

Hilbert  space  of  LgEO.l]  elements,  denoted  Ky.  consisting  of  range(R^) 
together  with  the  inner  product 

(Wv  = 


where  (f  }  and  (u  ,  n>l}  are  the  non-zero  eigenvalues  and  associated 
n  n 

orthonormal  eigenvectors  of  Ry  and  <*.*>  is  the  LgCO.l]  inner  product.  For 

M 

Lo[T<]>  1  i  i  i  M,  already  defined.  H  will  denote  ®  L0[T. ] .  Since  it  is 
Z  i  i=l  Z  1 

assumed  that  support[p^]  =  Lo[0.1],  the  eigenvalues  (X  )  of  are  all  non- 

zero  and  the  associated  orthonormal  eigenvectors  are  complete  in  LgfO.l], 

Let  A  be  a  positive  real-valued  random  variable  independent  of  the 

process  G,  and  define  (N^)  =  ( AG ^ ) .  (N^)  has  sample  paths  almost  surely  in 

^[O-l].  and  its  path-map  N  defines,  when  taking  equivalence  classes,  a 

afc/lBfLgCO,  1])  measurable  map,  where  is  the  Borel  a-field  of  LgfO.l], 

2 

Although  EN^  is  not  assumed  to  be  finite,  so  that  (Nt)  may  not  have  a 
covariance  function,  one  can  nevertheless  consider  the  conditional  RKHS 

=  Hq  a.s.,  since  A  >  0  a.s.  We  assume  that  f3 
is  the  smallest  a-field  containing  g(N):  P  =  a^(N). 

(B^)  is  the  Gaussian  mixture  local  martingale  defined  by  (B^)  =  (AGt). 
(Nt)  has  the  representation: 


of  (N t )  given  A.  Note  that 


N(u,  t)  =  f  <F 
J0 


<E(t.y) ,B(u,dy)> 


The  fact  that  this  is  derived  from  the  proper  canonical  representation  for 
(Gf)  implies  the  following  for  F: 

M 

Let  (Ptf)(s)  =  I  tl(s)f(s).  L^Tj]  =  p  yr,].  L*[T]  =  ®  L*[r  ]. 

L  ’  J  i  =  l 

Then: 
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a.  {Fi(s,*),  s  <  t}  spans  L^tl^]. 

P-  {E(s .  * )  .  s  ^  t}  spans  L^T] . 

Since  each  (G^  £)  is  Gaussian  and  has  orthogonal  increments,  it  is  a 
continuous  Lg-mart ingale  WRT  to  the  ^-algebras  generated  by  (Nt)  and  (B^)  and 
the  sets  of  measure  zero. 

It  is  assumed  that  (S^)  has  paths  which  are  a.s.  in  the  RKHS  of  (Gt)  and 
that  it  is  adapted  to  the  filtration  o(N) .  Then,  if  Yf  =  S  +  Nt.  a.s.  for  all 
fixed  t.  Y  has  the  representation  [4] 


Y( 


w.t)  =  f  <F(t,y)  ,X((j,dy)> 
J0  Ik 


where  X(u,t)  =  X^r(dx)s(w,x)  +  B(u,t)  and  s  is  such  that  S(w,t)  = 
/q<E( t.y) .r(dy)s(w,y)> 


Let  the  eigenvalues  and  associated  c.o.n.  eigenvectors  of  RG  be  denoted 
by  (Xn)  and  {e^,  n  2  1}-  Let  m:  L2[0,l]  -*  be  defined  by  m^(h,t)  = 

2  ..<H.  ,  e  Xh.e  >/X  where  H.  [y]  =  /«F.  (y.u)r  (du) .  One  then  has: 

Proposition  5:  Let  i)y  and  be  the  measures  induced  on  (IR*,9t*)  by  Y  and  N 
respectively,  and  let  Py  and  be  the  measures  induced  on  (L^fO.l], 
©(L^CO.l]))  by  the  same  processes.  Then 

a)  A1-A7  imply  Uy  <<  v ^  and  Py  <<  p^; 

^Y  dFX  r  1 

b)  Furthermore  ^~{h]  =  g— ■  J^m(h) J  p^-almost  every  h  in  L2[0,l]. 

Proof : 

a)  This  result  is  a  consequence  of  Proposition  4  above  and  Theorem  1  of 

[4]- 


b)  To  obtain  the  Radon-Nikodym  derivative,  one  writes: 
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Now. 


P(B  €  Cj,  N  €  C2}  =  P(AG  €  Cj .  AG  €  Cg} 

=  J  PA(da)  P(aC  £  Cj ,  «t  (  Cj) 

=  J^P*<da>  Jc  PaG(dh)  P{aG  €  Cj |aG  =  h}. 

from  [4],  we  have  that  P{aG  €  |aG  =  h}  is  a  point  mass 


concentrated  at  the  function  m  (h,*)  with  components 


m 


a(h,t)  :=  2  .ea><h.ea>/Xa. 

il  '  ,  i ,  t  n  n  n 

n=l 


where,  if  is  the  covariance  operator  of 


aG, 


-a  a  .a  a 
R0e  =  A  e  , 
G  n  n  n 


and  Hf  (x)  =  J*^F  (x,y)a2r  (dy)  =  a^H.  ,(x).  Now  Ra  =  a^-,  since  R^e  =  A  e  , 
i »  i  ui  i  i«t  b  b  ijrnnn 

and  Aa  =  a^A  .  Consequently, 


a 

e  =  e 
n  n 


m  (h, t)  =  m(h, t) 


Thus  for  all  a 


so  that 


P{aG  €  Cj|aG  =  h}  =  !<,  {s[h])}. 


P{B€C1|N=h}  =  Ic  {mJXJ}. 


An  explicit  representation  of  dpy/dp^  based  on  Proposition  5  requires  a 
representation  of  dP^/dPg.  But  rewriting  S  to  be  in  the  RKHS  of  N,  and 
applying  Proposition  4  yields : 


AC  &  MI  for  GMs  -  9/27/89  -  14 


exp 


U  <(s*(c.y)/-T2(c,y)).  E(c.dy)> 

-  if  us *(£., 


y)/Tr(c,y;;.  /'(dy)  (s  (c,y)/-r(c.y)  )> 


irmJ 


This  requires  of  course  that  (X^. )  have  a  stochastic  differential  equation 
expression,  but  this  can  be  obtained  as  indicated  in  the  Remark  following 
Proposition  4. 


Remark .  Proposition  3  and  the  preceding  discussion  show  that  Theorem  2  of  [4] 
can  be  adapted  to  yield  a  necessary  condition  for  absolute  continuity. 


4.  Computation  of  Mutual  Information  For  the  Channel 
With  Feedback  and  Spherical lv-Invarlant  Noise 

4.1.  Channel  Model 

We  now  proceed  to  derive  an  expression  for  the  mutual  information  of  a 
communications  channel  with  feedback,  where  the  noise  is  additive  and 
spherical ly-invariant .  We  begin  with  a  precise  description  of  the  channel. 

The  noise,  channel  output,  and  message  probabilities  will  all  be  defined  on 
(IR*,#*),  and  all  will  be  denoted  by  capital  P’s:  P^.  Py.  Pj.  The  joint 
measure  defined  by  (J.Y)  and  its  product  measure  is  then  defined  on  and 

denoted  by  Pj  „  and  Pj®Py 

a.  Assumptions  on  the  noise.  As  in  Section  3,  we  assume  that  N  is  m.s. 

continuous,  vanishes  a.s.  at  t  =  0  and  has  a  Cramer-Hida  representation  of 

2 

multiplicity  M  <  «*.  We  also  assume  that  EA  =1. 

b.  Assumptions  on  the  message.  The  message  is  a  real-valued  stochastic 

process  (Jt).  defined  on  (0,0).  and  independent  of  (w.r.t.  P  measure). 

Since  at(N)  =  crt(B)  =  ct(A)  v  a^(G)  for  t  €  [0,1]  because  of  the  proper 
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canonical  represention,  (Jt)  is  independent  of  A  and  independent  of  (G^). 

c.  Assumptions  on  the  channel.  Let  $  denote  the  family  of  maps 
2 

{_'■  [0,1]  -*  R  ,  and  let  9  be  the  filtration  of  §  generated  by  the  evaluation 
maps.  S  is  the  transmitted  signal,  and  Y  is  the  channel  output.  £  will  denote 
a  two-component  vector  with  elements  'f'j  =  J  and  ^  =  Y,  such  that  S(u,t)  = 
T[+(w)](t)  for  u  in  H,  t  in  [0,1].  T:  $x[0,l]  -»  R.  The  following  assumptions 
are  also  made: 

1.  T  is  adapted  to  9. 

2.  For  each  fixed  t  in  [0,1],  Y(<j,t)  =  T[tf[(u)](t)  +  N(w,t)  a.e.  dP(«). 

3.  T[i(u)]  €  RKHS(N) ,  a.e.  dP(u) . 

4.  EIIT[i(u)]ll^  <  ®.  where  llxll^  is  the  norm  of  x. 

5.  Y  is  adapted  to  <Z°(J)  v  ct°(N). 

6.  P  ~  PN  a.e.  dP.(v),  where  P  is  the  probability  on  (R*,3l*)  defined  by 

yV  n  j  yV 

the  process  Y  when  J  is  fixed  and  J  =  v. 

4.2.  Discussion 

Some  important  aspects  of  the  mathematical  and  physical  problems  are 
reflected  in  our  choice  of  constraints.  First,  the  Cramer-Hida  representation 
is  well-known  to  be  very  difficult  to  determine,  in  general.  Thus,  any 
reasonable  and  useful  set  of  constraints  should  not  assume  knowledge  of  this 
representation.  Similarly,  although  the  covariance  of  N  can  be  reasonably 
assumed  to  be  known,  the  probability  distribution  of  A  is,  for  practical 
purposes,  difficult  or  impossible  to  determine.  In  applications,  a  mapping  T 
as  above  will  be  used.  Thus,  we  state  all  assumptions  in  terms  of  this  T,  the 
obvious  measurability  properties  of  Y,  and  the  RKHS  of  N.  We  do  assume  finite 
Cram^r-Hida  multiplicity,  because  much  of  our  work  is  based  on  [4]. 
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Assumptions  3  and  4  are  made  to  ensure  that  the  capacity  will  be  finite. 

That  is,  in  the  Gaussian  channel  without  feedback,  a  necessary  and  sufficient 

o 

condition  for  finite  mutual  information  is  that  EIIS[<j]II^  <  »  [1],  Thus,  if 
such  a  condition  is  not  imposed  on  the  transmitted  signal,  then  for  almost  all 
a,  one  has  that  the  mutual  information  of  (S,  S+aG)  is  infinite  without  using 
feedback,  and  it  follows  that  the  mutual  information  of  (S,  S+N)  is  infinite. 

A  few  remarks  may  be  useful  in  order  to  give  some  perspective  to  the 
results  obtained  here,  including  a  contrast  with  other  versions,  in  particular 
those  of  Kadota,  Zakai  and  Ziv  (KZZ)  [15]  and  those  of  Hitsuda  and  Ihara 
(HI)  [12]. 

KZZ  assume  the  noise  in  the  channel  is  the  Wiener  process,  HI  that  it  is 
a  Gaussian  process  with  a  Cramer-Hida  representation  for  which  the  variances 
are  absolutely  continuous  with  respect  to  Lebesgue  measure.  Although  this 
assumption  is  not  explicitly  stated  by  HI,  it  seems  necessary  to  their 
development  (cf.  Prop.  1  of  [12]).  It  would  usually  be  extremely  difficult  to 
verify.  In  the  present  paper  it  is  assumed  that  the  noise  process  is 
spherically  invariant,  that  is,  a  scale  mixture  of  Gaussian  processes.  The 
basic  idea  of  the  HI  paper  is  to  extend  the  KZZ  result  to  any  Gaussian  process 
having  purely  continuous  Cramer-Hida  spectral  representation  by  using  the 
spectral  representation  in  conjunction  with  Girsanov’s  theorem.  That  approach 
is  also  used  here.  A  restriction  on  the  spectral  multiplicity  of  the  noise 
has  been  introduced,  for  two  reasons:  there  is  no  stochastic  calculus  for 

CO 

martingales  in  R  ,  and  there  has  been  no  attempt  on  our  part  to  extend  by 
other  means  the  formula  for  mutual  information  to  the  case  of  infinite 
multiplicity.  KZZ  assume  that  the  transmitted  signal  can  be  given  an  explicit 
functional  representation  on  the  space  obtained  as  the  product  of  the  message 
and  the  noise  space.  HI  assume  only  smoothness  and  adaptation  properties. 
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A  necessary  condition  for  finite  mutual  information  between  the  message 
and  the  channel  output  is  that  the  joint  measure  of  message  and  output  be 
absolutely  continuous  with  respect  to  their  product  measure.  Having  a 
functional  representation  for  the  transmitted  signal,  KZZ  can  give  conditions 
on  the  signal  that  insure  this  absolute  continuity.  This  is  no  longer 
possible  when  it  is  only  assumed  that  the  transmitted  signal  belongs  to  a 
particular  family,  the  RKHS  of  the  noise.  Thus,  one  must  assume  a  condition 
ensuring  the  absolute  continuity.  Such  an  assumption  is  not  stated  in  HI,  who 
instead  refer  to  the  Russian  version  of  [18].  However,  in  the  English  version 
of  [18],  we  are  unable  to  verify  that  Pj  ^  «  Pj®Py  using  only  assumption  3 
above  (the  assumption  used  by  HI),  or  even  with  the  stronger  assumption  4. 

Thus,  we  make  assumption  6.  It  will  be  seen  (Prop.  6  below)  that  this  is 
enough  to  give  Pj  ^  <<  Pj®Py.  Sufficient  conditions  for  P  y  ~  P^  a.e.  dPj(v) 

are  given  in  Section  3  above. 

In  addition  to  extending  the  HI  results  to  all  m.s.  continuous  Gaussian 

noise  processes  having  finite  Cramdr-Hida  multiplicity,  and  to  spherical ly- 

invariant  processes  with  finite  multiplicity,  we  also  use  a  substantially 

different  approach.  This  is  in  part  based  on  the  results  of  [4],  which  enable 

us  to  bypass  some  steps  in  the  HI  development  that  are  not  clear  to  us. 

As  in  Section  3,  there  exists  a  vector  process  (Xt)  having  paths  as.  in 

and  such  that  Y(u,t)  =  /I<E(t,x),  X(w,dx)>  M,  with  s  being  predictable, 

U  r"  ' 

square- in tegrab le ,  and  adapted  to  g(4[).  The  basic  approach  (as  in  HI)  is  to 

compute  the  mutual  information  I(J,Y)  by  showing  that  it  is  equal  to  I(J,X) 

and  to  then  work  with  the  latter.  It  is  thus  essential  that  Pj  ^  <<  Pj®P^, 

and  we  shall  also  need  the  fact  that  o(Y)  =  o(X). 


C* 
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Proposition  6.  Under  the  preceding  assumptions,  Pj  y  ~  Pj®Py.  PJ  x  ~ 
and  ct(Y)  =  g{X) . 


Remark .  The  proof  of  equivalence  of  the  measures  does  not  require  assumption 
c.4  of  Section  4.1. 


Proof .  Since  Y  is  adapted  to  2°(J)  v  2°(N).  there  exists  [10]  a  function 
g:  IR^xlR*  -»1R*,  measurable,  such  that  for  each  fixed  t  in  [0,1],  Y(w,t)  = 
irtg[J(w) >N(w)]  a.e.  dP(w).  If  YV  denotes  Y  when  J  is  fixed  and  J  =  v,  then 
YV(«,t)  =  Trtg[v,N(w)]  a.e.  dP,  each  fixed  t.  Thus  Py  =  [Pj®P^]og  1  and  P  ^  = 

P^°gy1 .  y  -»g(v.y).  From  Assumption  c.6  and  Theorem  1  of  [1],  this  gives 
Pj  y  ~  Pj®Py •  From  Theorem  7  of  [4],  and  proceeding  as  in  the  proof  of  Prop. 


T  J4  dPY  dPX 

5  above,  there  exists  m^:  R  -»  Cq.  measurable,  such  that  ^-(y)  =  jp-Km^y) 

a.e.  dP^(y)  and  (Theorem  6  of  [4])  Pg  =  P^om^  .  Thus,  for  A  in  <€^, 


dPv 


dP. 


PX(A)  =  SA  dP^y)dPB(y)  =  /  dP^5V)y>dPK(y> 

5  '  (A>  5 


=  / 
Eq1 (A) 


dPY 

dF<y)dPN(y)  =  p> 


N 


N' 


■y^^A). 


Thus-  PJ.Y  ~  PJ*PY  =>  Pj.X  ~  PJ®PX 


J1 

Jo 


To  show  that  o(Y)  =  o(X) .  we  again  note  that  (as  in  Section  3)  Y(u,t)  = 

<F(t,x),  X(u,dx)>  y  where  X(u,t)  =  r(dx)s(w,x)  +  B(w.t)  with  s 
r"  0 


predictable,  square-integrable  and  adapted  to  a(£) .  This  representation  of  Y 
yields  g(Y)  c  ct(X) . 

To  prove  that  g(X)  C  ct(Y) .  it  is  sufficient  to  show  that  the  closed 

linear  span  of  (X  .  s£t}  is  contained  in  span(Y  ,  sit)  for  all  t  €  [0.1].  For 
s  s 

this,  let  It  be  the  indicator  of  [0,t],  let  6  be  any  element  of  and  set 
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n 


f^x)  =  lt(x)0  -  I  aiE(si.x).  where  Sj  i  i  ...  i  sn  i  t  and  (a^  .  i£n}  is 
one  set  of  n  real  scalars.  Then 

n  pi  n 

<0.  X(u,  t)>  -  2  a  Y(<j,  s  )  =  <[I  (x)0-  2  a.F(s..x)],  X(u.dx)>  u 

BT  i=l  1  1  J0  1  1=1  1  1  r" 

=  J^<£n(x)  *  r(dx)s(a,.x)>^  +  J^fx).  B(u,dx)>  M. 


Thus . 


n 

E{<0.X( • ,  t)>  -  2  a  Y(  * ,  s  )}■“ 

r  i=i  1 


*  2{ElC<£*(x)- 


i2 


f(dx)s( * ,x)> 


*E|I0<VX>-  !*<••*'>>]  }■ 


Now.  using  JMI^  to  denote  the  norm  in  RKHS(N), 


1  2 

ELf0<£n<x>- 

S  e[T  ^(x).  rjdxj^lx))  j'<s(-,x).  r(dx)s(-.x)>  ] 

■  EI|£„“4(r),,I0«']|lN  *  "£r,"L2[r]E"I”«-JllN 

Since  ECJq^x)  .  B(  •  .dx)^]2  =  llfjl2  it  follows  that 

E{<0.X(-.t)>^(  -^(‘.Sj)}2  S  "fn^r]^  +  E||T°!£C*3IIn>- 

Since  F  arises  in  a  proper  canonical  representation,  (a^ . an)  and 

(Sj . s^)  (where  s^  £  t,  1  $  i  $  n)  can  be  chosen  so  that 


lim  ll£_  II 
n 


*»Vn '  °- 


This  completes  the  proof  that  g(Y)  =  g(X) . 
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We  now  have  that  I(J,Y)  =  I(J,X).  Thus,  the  computation  of  I(J,Y)  is 
reduced  to  that  of  I(J.X)  and  the  latter  requires  an  explicit  expression  for 


D 


.  dPJ.x 


J.x  "  dPj®dPx 


4.3.  Computation  of  Dj  ^ 

In  this  section  we  adopt  the  procedure  used  by  KZZ,  and  must  therefore 
"lift"  the  problem  as  defined  on  the  space  fi  to  the  product  of  the  path  space 
of  message  and  noise.  Consequently,  this  computation  is  based  on  the 
following  formula,  which  takes  into  account  that  J  and  B  are  independent 
(assumption  4.1.b). 


(dPj.x/dPj,B><J-*>  (dPj.x/dPj.BHJ-^  _  dj.x.B(j'x) 

(dP^dPg)(X)  ■  E^g(X) 


DJ  X(J.X)  -  (dpjddPx/dPj  B)(j.x) 


is 


(a)  Representation  of  (Xt).  Dj  ^  g  is  obtained  by  showing  that  it 
equal  to  (dP^|j/dPg)  =  Dj  x  3-  where  Px|j  ls  the  conditional  law  for  X  given 

Q 

J.  and  then  by  computing  D.  v  _.  This  is  done  by  "lifting"  the  equation  for  X 
onto  product  space  in  order  to  obtain  a  "decoupling"  of  the  effect  of  J  and  N. 
To  do  this,  one  must  give  an  explicit  functional  representation  for  (Xt). 

Let  A  be  the  family  of  maps  g:  [0,1]  -*  where  g^  €  IR^’^  and 

g^  €  C0[0.1],  1  i  i  i  M.  is  the  filtration  of  A  generated  by  the  evaluation 

maps.  (Xt)  will  denote  the  vector  stochastic  process  with  components  =  J, 
and  t)  =  (Xj  t)  for  1  £  i  i  M.  (X^)  then  has  the  representation,  a.e. 
dP(w) . 


X(W.t)  =  f 

Jo 


+  B(w,  t) 


for  each  fixed  t.  t  is  ^-predictable  [19].  In  fact,  we  know  that 


AC  4  MI  for  GMs  -  9/27/89  -  21 


i)  (s^)  is  2(i) "Predictable. 

1 i )  g(i)  =  J)  V  g(Y). 
iil)  a(Y)  -  g(X) . 

Thus,  (§t)  is  o(x)  predictable,  so  that  (s^.)  cannot  be  distinguished 

o 

(pathwise)  from  a  process  which  is  a  (x)-predictable;  s  can  be  written  as 

s(w,t)  =  i(x[w],t)  ([4,  Lemma  2]  and  [19,  p.  67]). 

(b)  Representation  of  X  on  function  space.  Let  and  q^  be  the 

projections  onto  the  first  and  second  components  of  $.  On  9^  (defined  in 

4.1.c),  consider  the  probability  Pj®P^,  induced  by  0:  0x[O,l]  -»  IR^  with 

components  0^  =  J  and  0 ^  s  N.  Let  be  the  completion  of  9^  with  respect  to 

PT®PM,  with  P f®PM  the  extension  of  P .®PU  to  9  .  Finally,  let  9  be  the 
JN  JN  J  IH  1  t 

filtration  generated  by  and  the  sets  of  9 ^  whose  Pj®P^  measure  is  zero  9 
will  then  denote  the  resulting  filtration  of  $. 

With  these  definitions,  there  exist  processes  X^.  B^,  and  defined  on 
the  probability  space  ($,  9^ .  Pj®P^)  which  are  adapted  to  9  and  which  have  the 
following  properties. 

(1)  t)  is  a  continuous  local  martingale  such  that  (a.e.  dP(w),  all  t  in 

[0.1]): 

-  B^(0[(j],t)  =  B(w.t) 

-  (B^  t)  depends  on  (Nf)  only  (not  on  (J£)) 

-  <B^>(£ . c)  =  A^(f)r(t) 

-  A^(0[u])  =  A(w) 

-  A^  depends  on  (N£)  only  (not  on  (J^)). 

(2)  X,(0[«].t)  =  X(a).t). 

(3)  Xj(£ .  t)  =  Sq  r(dx)r^[[^](f.x)]  +  B^(f.t) 

f  qi  ] 

where  is  predictable  for  9  and  ^  ]  °  0[<*>],tJ  =  i(^[u].t)  =  s(w,t). 
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The  validity  of  these  statements  is  seen  as  follows.  (X^.)  is  continuous 

and  adapted  to  a(Y),  so  that  it  is  adapted  to  g(J)  v  g(N).  (X^)  *s  thus 

predictable,  so  that  it  cannot  be  distinguished  (pathwise)  from  a  process  that 

o  o 

is  predictable  for  a  (J)  v  a  (N)  [17].  It  then  has,  as  in  (a),  a  functional 
representation  ( lemma  2  of  [4])  as  given  in  (2)  above.  The  same  argument 
applies  to  (B^)  and  t.  Moreover,  since  A(g>)  =  "'r(B[(i)] ,  t) ,  t  >  0,  is  given 
by  A4U)  =  T^CfJ.t).  t  >  0. 

(c)  Computation  of  PT  v  D.  From  the  definitions,  one  has 

Pj  ^[KxC]  =  PJ8N{[KxlRI]  n  [X^  €  C]}  =  |  P  T(du)PN{v:  X^]  €  C} . 

K 

This  shows,  in  particular,  that  P^°[X^]  *  is  a  version  of  the  conditional  law 
of  X  given  J  =  u.  where  X^(v)  =  X^]. 

Now.  since 

1)  • 


by  fixing  the  first  component,  u.  in  the  argument  of  X^,  one  has  a  stochastic 
differential  equation  whose  solution,  X^,  has  law  P^°[X^]  *  absolutely 


continuous  with  respect  to  P^°B^ ,  which  means  that  P 
continuous  with  respect  to  Pg.  Thus 


X  |  J=u 


is  absolutely 


dP. 


-J 

JKxC 


dPY  . 

•  ti  -I  v  X  o  — U  J 


dPJ.B^c^-d^<^ 

O 


Moreover , 
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dP, 


dP 


Xjj=u 


B 


<c)  =  exp- 


.UU  U0<I*([c]-x)-  I(£,dx)> 


-  *£<!*<[“]. *>.  r(dx)lt([“].x)>ffiH' 


dP. 


A  I  ~U 

Note  that  h-’  (u.c)  — ►  — t1J~U(c)  is  product-measurable  [22]. 

5 

(d)  Computation  of  D^.  One  begins  again  with  the  representation 

X(cj,t)  =  f  r(dx)x(^[u]  ,x)  +  B(cj.t). 

J0 

*ri 

Let  t  have  components  T ^  ^r- ,  1  i  i  £  M.  Assumption  4.1.c  then  yields 


F.f  r  (dx)llT(x[<j].x)ll2u  <  “. 
J0  1  ST 


Using  Proposition  3  and  Memin  [19],  one  obtains  that  (X^)  has  a  representation 
of  the  form 


X(w.t)  =  J  adxji^XM.x)  +  BX(w.t) 


where:  t^,  is  predictable  for  ^ ; 

Ep  [  <!<•>(£, x),  T(dx)  Ip(c,x)>  „  <  »; 
XJ0  ^  I r 


(8t)  is  a  spherically  invariant  martingale  for  ct(X)  such  that 


X  2 
<B  >  =  AT. 


Consequently,  with  respect  to  P^, 
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<I~(c.x).  JI(c.dx)> 


(c,  1)  u0 


r1  ii 

-  <!(>(£. x).  rfdxjT^-fc.x))  Mjj 


where  l~(X[>],t)  is  a  version  of  the  conditional  expectation 
E[lU[-].t)|at(X)]. 


4.4.  Expression  for  I(J.X) 

One  has  by  the  change  of  variable  formula, 


I(J.X)  =  J  1 
r[0-1]xc|J[0.1] 


=  E  ln{ 


dpJwx 


lX(-)J 


-  E  -  E  ln{DX.B(K-))}' 

No.  in  Dj ,x.b[[^(“)]]  =  r2(gu]  ,,  J^1*  ([>«"!] 'X] '  X<“'dx>>RK 

‘  MJ0<z*[[x(“)]  x]-  E(dx»^[^)]-]> 

Now  t2(X|>,1])  =  ^2(B[w],1)  =  A2(u).  and  ^[[^["jj.t]  =  S(«.t).  so  that 

>"  DJ.x.s([x(«)]]  =?{Jo<s("-x,-S(“',ix)>  '  4o<s("-!t)'r(dx,s(u'x)V 

=  -^ISII2  +  Kj  f  <s(w,x)  ,B(w,dx)>  „  . 

2A2  N  A2  J0  or 


Simi lar ly , 


ln  =  ^li#N  *  ^Io4<"-x)-8(“-dx)>- 
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where  S  and  s  denote  appropriate  versions  of  the  conditional  expectations  of  S 
and  s  respectively,  the  filtration  being  g(X) . 

So,  under  the  assumption  that 

e 

one  has 

I(J.Y)  =  *(e  ^[IISII2  -  IISII2]j. 

From  this  expression,  an  upper  bound  for  the  mutual  information  of  the 

12  2 
feedback  channel  is  'A  E  — gllSIl^.  If  the  signal  is  contrained  by  EIISII^  <  P  , 

A 

then  an  upper  bound  on  the  capacity  of  the  no-feedback  channel  (S  and  A 

independent)  is  obviously  if  PqE(— i) .  This  is  also  a  lower  bound  [3],  so  that 

A 

the  no-feedback  capacity  (subject  to  EIISII2  £  Pq)  is  if  PqE(-^-)  .  The  lower 

bound  can  be  shown  in  several  ways.  For  example,  one  can  note  that  it  is 
sufficient  to  consider  I(J.X),  where 

X(w,t)  =  f  J(u,s)ds  +  A(<j)W(<j,  t) , 

J0 

E  s)ds  i  P  and  (W^)  is  the  standard  Wiener  process.  IIXII2  = 

12  n 

JqX  (t)dt.  Choosing  a  sequence  (Jt)  of  zero-mean  Gaussian  processes 

independent  of  A  and  (Wt).  with  (J^)  having  covariance  function  P^e  SL 

one  can  show  that  EIIJnll2  =  Pn  while  lim  EIIJnll2  =  0.  That  is,  from  [15], 

u  n 

E[  II Jnll2  | A]  $  n_1PQ  a.  s .  ,  so  that  EIIJnll2  -►  0. 
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